Subsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf. (ˆθn θ (P ) under P. For real ˆθ n, (τ n
|
|
- Arron Sherman
- 5 years ago
- Views:
Transcription
1 Subsamlig vs Bootstra Dimitris N. Politis, Joseh P. Romao, Michael Wolf R x, θ P = τ ˆθ θ P Examle: ˆθ = X, τ =, θ = EX = µ P orˆθ = mi X, τ =, θ P = su{x : F x 0} Defie: J P, the istributio of τ ˆθ θ P uer P. For real ˆθ, J x, P P rob P τ ˆθ θ P x Sice P is ukow, θ P is ukow, a J x, P is also ukow. The bootra estimate J x, P by J x, ˆP, where ˆP is a cosistet estimate of P i some sese. For examle, take hatp x = i= X i x the emirical istributio: su x ˆP x P x 0 Similarly estimate αth quatile of J x, P by J x, ˆP : i.e. Estimate J x, P by x, ˆP. J Usually J x, ˆP use Mote Carlo aroximatio: for ˆθ,i = ˆθ X,i,..., X,i. a.s. ca t be exlicitly calculatealthough i some simle case it ca be, J x, ˆP B B i= τ ˆθ,i ˆθ x Whe bootstra worksthe meaig of works, for each x, J x, ˆP J x, P α, ˆP = J J α, P 0 0 Whe shoul Bootstra work? Nee local uiformity i weak covergece:
2 . Usually J x, P J x, P. 2. Also usually ˆP P a.s. i some sese, say su x ˆP x P x 0. a.s. 3. Suose for each sequece P s.t. P P, say su x P P 0, it is also true that J x, P J x, P, the it must be true that a.s. J x, ˆP J x, P 4. So it es u havig to show for P P, J x, P J x, P, use triagular array formulatio. Case whe it works: samle mea with fiite variace. It is kow that:. su x ˆF x F x 0. a.s. 2. θ ˆF = i= X a.s. i θ F = EX. 3. σ 2 ˆF = i= Xi X 2 a.s. σ 2 F = V arx. 4. Use Lierberg-Feller for the triagular array, alie to the etermiistic sequece of P such that: su x P x P x 0; 2 θ P θ P ; 3 σ 2 P σ 2 P, it ca be show that X θ P N 0, σ 2 uer P. 5. Sice ˆP satisfies,2,3 a.s., therefore a.s. J x, ˆP J x, P. Therefore local uiformity of weak covergece is satisfie here. Cases whe bootstra fails:. Orer Statistics: F U 0, θ, a X,..., X is the orer statistics of the samle, so X is the maximum: P θ X θ The bootstra versio: > x = P X < θ θx = P P X X /X = 0 = X i < θ θx = = θ x e 0.63 θ θx e x 2
3 2. Degeerate U-statistics: Take w x, y = xy, θ F = w x, y F x F y = µ F 2. If µ F 0 it is kow that The bootstra works. ˆθ = θ ˆF = X i X j i j S x = xyf y = xµ F ˆθ θ N 0, 4V ar S X = N 0, 4 µ 2 EX 2 µ 4 But if µ F = 0 = θ F = 0: θ ˆF = X i X j = X 2 i j θ ˆF θ F = X 2 S 2 N 0, σ 2 σ 2 [ However the bootstra versio of θ ˆF ] θ ˆF : i Xi X 2 = X2 S2 [ X 2 ] [ S 2 X 2 ] S2 2 = X S 2 X 2 + S 2 X 2 X 2 = [ X X ] X X X N 0, σ N 0, σ 2 X Subsamlig: ii case: Y i block of size b from X,..., X, i =,..., q, for q = b. Let ˆθ,b,i = ˆθ Y i calculate with the ith block of ata. Use the emirical istributio of τ b ˆθ,b,i ˆθ over the q seuo-estimates to aroximate the istributio of τ ˆθ θ : Aroximate by J x, P = P τ ˆθ θ x L,b x = q i= τ b ˆθ,b,i ˆθ x 3
4 Claim: If b, b/ 0, τ b /τ 0, as log as τ ˆθ θ somethig, J x, P L,b x 0 Differet motivatio for Subsamlig vs. Bootstra: Subsamlig: each subset of size b comes from the TRUE moel. Sice τ ˆθ θ J x, P, so as log as b : τ b ˆθb θ J x, P For large, the istributios of τ ˆθ θ a τ b ˆθb θ shoul be close. But Sice τ b ˆθb θ = τ b ˆθb ˆθ + τ b ˆθ θ τb τ b ˆθ θ = O τ = o The istributios of τ b ˆθb θ a τ b ˆθb ˆθ shoul be close. The istributio of τ b ˆθb ˆθ is estimate by the emirical istributio over q = b seuo-estimates. Bootstra: Recalculate the statistics from the ESTIMATED moel ˆP. Give that ˆP is close to P, hoefully J x, ˆP is close to J x, P Or to J x, P, the limit istributio. But whe bootstra fails ˆP P J x, ˆP J x, P Formal Proof of cosistecy of subsamlig: Assumtios: τ ˆθ θ Nee to show: L,b x J x, P 0. Sice τ θ θ J x, P,b, b 0, τ b τ 0. 0, it is eough to show U,b x = q i= τ b ˆθ,b,i θ x J x, P U,b x J x, P = U,b x EU,b x + EU,b x J x, P 4
5 Eough to show U,b x EU,b x 0 a EU,b x J x, P 0 But EU,b x J x, P = J b x, P 0 U,b x is a bth orer U-statistics with kerel fuctio boue by,. Use Hoeffig exoetial-tye iequalityserflig980, Thm A. 20: P U,b x J b x, P ɛ ex 2 b ɛ2 / [ ] = ex b t2 0 as b So. L,b x J x, P = L,b x U,b x + U,b x J b x, P + J b x, P J x, P Q.E.D. Time Series!: Resect the orerig of the ata to reserve correlatio. ˆθ,b,t = ˆθ b X t,..., X t+b, q = T b L,b x = q i= τ b ˆθ,b,t ˆθ x Assumtio: τ ˆθ θ J x, P, b, b 0, τ b τ 0, α m 0. Result: L,b x J x, P 0. Most ifficult art: To show τ ˆθ θ J x, P. Ca treat ii ata as time series, or eve usig o-overlaig blocks k = [ ] b, but usig b more efficiet. For examle, if Ū x = k k j= τ b [R,b,j θ P ] x the U,b x = E [ Ū x X \ ] = E [ τb [R,b,j θ P ] x X ] for X = X,..., X. U,b x is better tha Ū x sice X is sufficiet statistics for ii ata. 5
6 Hyothesis Testig: T = τ t X,..., X, G x, P = P rob τ x P P 0 J x, P Ĝ,b x = q T,b,i x = q τ b t,b,i x i= i= As log as b, b 0, the uer P P 0: Ĝ,b x G x, P If uer P P, T, the x, Ĝ,b x 0. Key ifferece with cofiece iterval: o t ee τ b τ θ 0 but assume kow uer the ull hyothesis. 0, because o t ee to estimate Estimatig the ukow rate of covergece: Assume that τ = β, for some β > 0, but β is ukow. Estimate β usig ifferet size of subsamlig istributio. Key iea: Comare the shae of the emirical istributios of ˆθ b ˆθ for ifferet values of b to ifer the value of β. Let q = b for ii ata, or q = T b + for time series ata: This imlies L,b x τ b q L,b x q a= τ b ˆθ,b,a ˆθ x a= ˆθ,b,a ˆθ x L,b x τ b = L,b τ b x t Sice L,b x τ b Same as x = L,b t τ b = τ b τ b x = τ b L,b t J x, P, if J x, P is cotiuous a icreasig, it ca be ifere that L,b t τ b = J t, P + o τ b L,b t = J t, P + o 6
7 So b β L,b t = J t, P + o take logassumig J t, P > 0, or t > J 0, P, for ifferet b a b 2, the this becomes Differet out the fixe effect So estimate β by β log b + log L,b t = log J t, P + o β log b 2 + log L,b 2 t = log J t, P + o β log b log b 2 = log L,b 2 t log L,b t + o ˆβ = log b log b 2 log L,b 2 t log L,b t = β + log b log b 2 o Take b = γ, b 2 = γ 2, γ > γ 2 > 0 ˆβ β = γ γ 2 log o = o log How to kow t > J 0, P So estimatig J 0, P ot a roblem. L,b 0 τ b = L,b 0 = J 0, P + o Alteratively, take t 2 0.5,, take t 0, 0.5 b β L,b t 2 L,b t = J t 2 P J t P + o β log b + log L,b t 2 L,b t = log J t 2 P J t P + o ˆβ = log b log b 2 [ log L,b 2 t 2 L,b 2 t log L,b t 2 L,b t ] Take b = γ, b 2 = γ 2, > γ > γ 2 > 0, As before ˆβ β = o log 7
8 Two ste subsamlig: ˆτ = ˆβ L,b x ˆτ b = q a= ˆτ b ˆθ,b,a ˆθ x Ca show that su x L,b x ˆτ b J x, P 0. Problem: imrecise i small samles. E.g. i variatio estimatio, best choice of b gives error rate of O /3 but arameter estimates, if moel is true, gives O /2 error rate. Bootstra ivotal statistics, whe alicable, gives eve better tha O /2 error rate. 8
Lecture 13: Subsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf
Lecture 13: 2011 Bootstrap ) R n x n, θ P)) = τ n ˆθn θ P) Example: ˆθn = X n, τ n = n, θ = EX = µ P) ˆθ = min X n, τ n = n, θ P) = sup{x : F x) 0} ) Define: J n P), the distribution of τ n ˆθ n θ P) under
More informationTHE INTEGRAL TEST AND ESTIMATES OF SUMS
THE INTEGRAL TEST AND ESTIMATES OF SUMS. Itroductio Determiig the exact sum of a series is i geeral ot a easy task. I the case of the geometric series ad the telescoig series it was ossible to fid a simle
More informationSummary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More information13.1 Shannon lower bound
ECE598: Iformatio-theoretic methods i high-dimesioal statistics Srig 016 Lecture 13: Shao lower boud, Fao s method Lecturer: Yihog Wu Scribe: Daewo Seo, Mar 8, 016 [Ed Mar 11] I the last class, we leared
More information1 Review and Overview
CS229T/STATS231: Statistical Learig Theory Lecturer: Tegyu Ma Lecture #12 Scribe: Garrett Thomas, Pega Liu October 31, 2018 1 Review a Overview Recall the GAN setup: we have iepeet samples x 1,..., x raw
More informationtests 17.1 Simple versus compound
PAS204: Lecture 17. tests UMP ad asymtotic I this lecture, we will idetify UMP tests, wherever they exist, for comarig a simle ull hyothesis with a comoud alterative. We also look at costructig tests based
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More information18. Two-sample problems for population means (σ unknown)
8. Two-samle roblems for oulatio meas (σ ukow) The Practice of Statistics i the Life Scieces Third Editio 04 W.H. Freema ad Comay Objectives (PSLS Chater 8) Comarig two meas (σ ukow) Two-samle situatios
More informationBasics of Inference. Lecture 21: Bayesian Inference. Review - Example - Defective Parts, cont. Review - Example - Defective Parts
Basics of Iferece Lecture 21: Sta230 / Mth230 Coli Rudel Aril 16, 2014 U util this oit i the class you have almost exclusively bee reseted with roblems where we are usig a robability model where the model
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationECE534, Spring 2018: Solutions for Problem Set #2
ECE534, Srig 08: s for roblem Set #. Rademacher Radom Variables ad Symmetrizatio a) Let X be a Rademacher radom variable, i.e., X = ±) = /. Show that E e λx e λ /. E e λx = e λ + e λ = + k= k=0 λ k k k!
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More information6.3.3 Parameter Estimation
130 CHAPTER 6. ARMA MODELS 6.3.3 Parameter Estimatio I this sectio we will iscuss methos of parameter estimatio for ARMAp,q assumig that the orers p a q are kow. Metho of Momets I this metho we equate
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationLECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if
LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationChapter 2 Transformations and Expectations
Chapter Trasformatios a Epectatios Chapter Distributios of Fuctios of a Raom Variable Problem: Let be a raom variable with cf F ( ) If we efie ay fuctio of, say g( ) g( ) is also a raom variable whose
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationJohn H. J. Einmahl Tilburg University, NL. Juan Juan Cai Tilburg University, NL
Estimatio of the margial exected shortfall Laures de Haa, Poitiers, 202 Estimatio of the margial exected shortfall Jua Jua Cai Tilburg iversity, NL Laures de Haa Erasmus iversity Rotterdam, NL iversity
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationSTAT-UB.0103 NOTES for Wednesday 2012.APR.25. Here s a rehash on the p-value notion:
STAT-UB.3 NOTES for Wedesday 22.APR.25 Here s a rehash o the -value otio: The -value is the smallest α at which H would have bee rejected, with these data. The -value is a measure of SHOCK i the data.
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationTesting for Convergence
9.5 Testig for Covergece Remember: The Ratio Test: lim + If a is a series with positive terms ad the: The series coverges if L . The test is icoclusive if L =. a a = L This
More informationBIOSTATISTICAL METHODS FOR TRANSLATIONAL & CLINICAL RESEARCH
BIOSAISICAL MEHODS FOR RANSLAIONAL & CLINICAL RESEARCH Direct Bioassays: REGRESSION APPLICAIONS COMPONENS OF A BIOASSAY he subject is usually a aimal, a huma tissue, or a bacteria culture, he aget is usually
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationSolutions to Practice Midterms. Practice Midterm 1
Solutios to Practice Midterms Practice Midterm. a False. Couterexample: a =, b = b False. Couterexample: a =, b = c False. Couterexample: c = Y cos. = cos. + 5 = 0 sice both its exist. + 5 cos π. 5 + 5
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationHybridized Heredity In Support Vector Machine
Hybridized Heredity I Suort Vector Machie May 2015 Hybridized Heredity I Suort Vector Machie Timothy Idowu Yougmi Park Uiversity of Wiscosi-Madiso idowu@stat.wisc.edu yougmi@stat.wisc.edu May 2015 Abstract
More informationFINAL EXAMINATION IN FOUNDATION OF ANALYSIS (TMA4225)
Norwegia Uiversity of Sciece ad Techology Departmet of Mathematical Scieces Page of 7 Cotact durig exam: Eugeia Maliikova (735) 52 57 FINAL EXAMINATION IN FOUNDATION OF ANALYSIS (TMA4225) Moday, December
More informationEconomics 326 Methods of Empirical Research in Economics. Lecture 18: The asymptotic variance of OLS and heteroskedasticity
Ecoomics 326 Methods of Empirical Research i Ecoomics Lecture 8: The asymptotic variace of OLS ad heteroskedasticity Hiro Kasahara Uiversity of British Columbia December 24, 204 Asymptotic ormality I I
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationEmpirical likelihood for parametric model under imputation for missing
Emirical likelihood for arametric model uder imutatio for missig data Lichu Wag Ceter for Statistics Limburgs Uiversitair Cetrum Uiversitaire Camus B-3590 Dieebeek Belgium Qihua Wag Istitute of Alied Mathematics
More informationNotes on the prime number theorem
Notes o the rime umber theorem Keji Kozai May 2, 24 Statemet We begi with a defiitio. Defiitio.. We say that f(x) ad g(x) are asymtotic as x, writte f g, if lim x f(x) g(x) =. The rime umber theorem tells
More informationp we will use that fact in constructing CI n for population proportion p. The approximation gets better with increasing n.
Estimatig oulatio roortio: We will cosider a dichotomous categorical variable(s) ( classes: A, ot A) i a large oulatio(s). Poulatio(s) should be at least 0 times larger tha the samle(s). We will discuss
More informationAsymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationTAMS24: Notations and Formulas
TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More information( ) = is larger than. the variance of X V
Stat 400, sectio 6. Methods of Poit Estimatio otes by Tim Pilachoski A oit estimate of a arameter is a sigle umber that ca be regarded as a sesible value for The selected statistic is called the oit estimator
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationLogit regression Logit regression
Logit regressio Logit regressio models the probability of Y= as the cumulative stadard logistic distributio fuctio, evaluated at z = β 0 + β X: Pr(Y = X) = F(β 0 + β X) F is the cumulative logistic distributio
More informationIntroduction to Probability and Statistics
Introduction to Probability and Statistics Chater 8 Ammar M. Sarhan, asarhan@mathstat.dal.ca Deartment of Mathematics and Statistics, Dalhousie University Fall Semester 28 Chater 8 Tests of Hyotheses Based
More informationFinal Solutions. 1. (25pts) Define the following terms. Be as precise as you can.
Mathematics H104 A. Ogus Fall, 004 Fial Solutios 1. (5ts) Defie the followig terms. Be as recise as you ca. (a) (3ts) A ucoutable set. A ucoutable set is a set which ca ot be ut ito bijectio with a fiite
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationResearch Article New Bandwidth Selection for Kernel Quantile Estimators
Hidawi Publishig Cororatio Joural of Probability ad Statistics Volume, Article ID 3845, 8 ages doi:.55//3845 Research Article New Badwidth Selectio for Kerel Quatile Estimators Ali Al-Keai ad Kemig Yu
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationTests of Hypotheses Based on a Single Sample (Devore Chapter Eight)
Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........
More informationUNIFORM RATES OF ESTIMATION IN THE SEMIPARAMETRIC WEIBULL MIXTURE MODEL. BY HEMANT ISHWARAN University of Ottawa
The Aals of Statistics 1996, Vol. 4, No. 4, 1571585 UNIFORM RATES OF ESTIMATION IN THE SEMIPARAMETRIC WEIBULL MIXTURE MODEL BY HEMANT ISHWARAN Uiversity of Ottawa This aer resets a uiform estimator for
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationThe Hong Kong University of Science & Technology ISOM551 Introductory Statistics for Business Assignment 3 Suggested Solution
The Hog Kog Uiversity of ciece & Techology IOM55 Itroductory tatistics for Busiess Assigmet 3 uggested olutio Note All values of statistics i Q ad Q4 are obtaied by Excel. Qa. Let be the robability that
More informationx iu i E(x u) 0. In order to obtain a consistent estimator of β, we find the instrumental variable z which satisfies E(z u) = 0. z iu i E(z u) = 0.
27 However, β MM is icosistet whe E(x u) 0, i.e., β MM = (X X) X y = β + (X X) X u = β + ( X X ) ( X u ) \ β. Note as follows: X u = x iu i E(x u) 0. I order to obtai a cosistet estimator of β, we fid
More informationEstimation with Overidentifying Inequality Moment Conditions Technical Appendix
Estimatio with Overidetifyig Iequality Momet Coditios Techical Aedix Hyugsik Roger Moo Uiversity of Souther Califoria Frak Schorfheide Uiversity of Pesylvaia, CEPR ad NBER August 27, 2008 Not iteded to
More informationLecture 13: Maximum Likelihood Estimation
ECE90 Sprig 007 Statistical Learig Theory Istructor: R. Nowak Lecture 3: Maximum Likelihood Estimatio Summary of Lecture I the last lecture we derived a risk (MSE) boud for regressio problems; i.e., select
More information1. Do the following sequences converge or diverge? If convergent, give the limit. Explicitly show your reasoning. 2n + 1 n ( 1) n+1.
Solutio: APPM 36 Review #3 Summer 4. Do the followig sequeces coverge or iverge? If coverget, give the limit. Eplicitly show your reasoig. a a = si b a = { } + + + 6 c a = e Solutio: a Note si a so, si
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationLast Lecture. Biostatistics Statistical Inference Lecture 16 Evaluation of Bayes Estimator. Recap - Example. Recap - Bayes Estimator
Last Lecture Biostatistics 60 - Statistical Iferece Lecture 16 Evaluatio of Bayes Estimator Hyu Mi Kag March 14th, 013 What is a Bayes Estimator? Is a Bayes Estimator the best ubiased estimator? Compared
More informationMA Advanced Econometrics: Properties of Least Squares Estimators
MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample
More informationElementary Statistics
Two Samle Mea Cha08 Dr. Ghamary Page Elemetary Statitic M. Ghamary, Ph.D. Chater 8 Tet of Hyothei a Cofiece Iterval for Two Samle Two Samle Mea Cha08 Dr. Ghamary Page Tet of Hyothei for Two amle: A Statitical
More informationStat 200 -Testing Summary Page 1
Stat 00 -Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More information= p x (1 p) 1 x. Var (X) =p(1 p) M X (t) =1+p(e t 1).
Prob. fuctio:, =1 () = 1, =0 = (1 ) 1 E(X) = Var (X) =(1 ) M X (t) =1+(e t 1). 1.1.2 Biomial distributio Parameter: 0 1; >0; MGF: M X (t) ={1+(e t 1)}. Cosider a sequece of ideedet Ber() trials. If X =
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More informationIn this section, we show how to use the integral test to decide whether a series
Itegral Test Itegral Test Example Itegral Test Example p-series Compariso Test Example Example 2 Example 3 Example 4 Example 5 Exa Itegral Test I this sectio, we show how to use the itegral test to decide
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationCommon Large/Small Sample Tests 1/55
Commo Large/Small Sample Tests 1/55 Test of Hypothesis for the Mea (σ Kow) Covert sample result ( x) to a z value Hypothesis Tests for µ Cosider the test H :μ = μ H 1 :μ > μ σ Kow (Assume the populatio
More informationHomework for 2/3. 1. Determine the values of the following quantities: a. t 0.1,15 b. t 0.05,15 c. t 0.1,25 d. t 0.05,40 e. t 0.
Name: ID: Homework for /3. Determie the values of the followig quatities: a. t 0.5 b. t 0.055 c. t 0.5 d. t 0.0540 e. t 0.00540 f. χ 0.0 g. χ 0.0 h. χ 0.00 i. χ 0.0050 j. χ 0.990 a. t 0.5.34 b. t 0.055.753
More informationSequences. Notation. Convergence of a Sequence
Sequeces A sequece is essetially just a list. Defiitio (Sequece of Real Numbers). A sequece of real umbers is a fuctio Z (, ) R for some real umber. Do t let the descriptio of the domai cofuse you; it
More informationof the matrix is =-85, so it is not positive definite. Thus, the first
BOSTON COLLEGE Departmet of Ecoomics EC771: Ecoometrics Sprig 4 Prof. Baum, Ms. Uysal Solutio Key for Problem Set 1 1. Are the followig quadratic forms positive for all values of x? (a) y = x 1 8x 1 x
More informationSolutions to Problem Sheet 1
Solutios to Problem Sheet ) Use Theorem. to rove that loglog for all real 3. This is a versio of Theorem. with the iteger N relaced by the real. Hit Give 3 let N = [], the largest iteger. The, imortatly,
More information1 Covariance Estimation
Eco 75 Lecture 5 Covariace Estimatio ad Optimal Weightig Matrices I this lecture, we cosider estimatio of the asymptotic covariace matrix B B of the extremum estimator b : Covariace Estimatio Lemma 4.
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.
More informationProposition 2.1. There are an infinite number of primes of the form p = 4n 1. Proof. Suppose there are only a finite number of such primes, say
Chater 2 Euclid s Theorem Theorem 2.. There are a ifiity of rimes. This is sometimes called Euclid s Secod Theorem, what we have called Euclid s Lemma beig kow as Euclid s First Theorem. Proof. Suose to
More informationA Central Limit Theorem for Belief Functions
A Cetral Limit Theorem for Belief Fuctios Larry G. Estei Kyougwo Seo November 7, 2. CLT for Belief Fuctios The urose of this Note is to rove a form of CLT (Theorem.4) that is used i Estei ad Seo (2). More
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationFinal Review. Fall 2013 Prof. Yao Xie, H. Milton Stewart School of Industrial Systems & Engineering Georgia Tech
Fial Review Fall 2013 Prof. Yao Xie, yao.xie@isye.gatech.edu H. Milto Stewart School of Idustrial Systems & Egieerig Georgia Tech 1 Radom samplig model radom samples populatio radom samples: x 1,..., x
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More informationMOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.
XI-1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI-2 (1075) STATISTICAL DECISION MAKING Advaced
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More information1 Probability Generating Function
Chater 5 Characteristic Fuctio ad Probablity Covergece Theorem Probability Geeratig Fuctio Defiitio. For a discrete radom variable X which ca oly achieve o-egative itegers, we defie the robability geeratig
More information